Create LLM Resources

LLM resources provide a centralized way to manage your Artificial Intelligence (AI) model connections within Druid. By creating LLM resources in the Druid Portal, you define the necessary credentials and endpoints once, allowing you to reuse them consistently across multiple features, including:

  • LLM NLU (Natural Language Understanding)
  • LLM Machine Translation
  • LLM Response Streaming

This unified approach simplifies configuration and ensures consistent access to your language models throughout your bot implementation.

NOTE: This feature is available in Druid 9.11 and higher.

Follow these steps to add and configure a new LLM connection:

  1. Go to Administration > LLM Resources.
  2. Click the Create LLM Resource button. The LLM Resource Details modal appears.
  3. Complete the LLM resource details:
    1. Client Type: Select the type of LLM client you are using.
    2. Model: Select or enter the specific model name you intend to use.
    3. API Url: Enter the LLM service endpoint URL.
    4. API key: Enter the secret API key provided by your LLM service provider.

    HINT: If you are using a Druid LLM resource (a model provided by Druid), you can find the necessary API Url and API key in the Druid Portal under Administration > Druid Services.

  4. If you selected Google as Client Type, you also need to enter the location of the location for a Vertex AI and your Google project id.
  5. Click Save & Close.